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Abstract 


Statistical  image  segmentation  refers  to  the  computer-oriented 
procedures  that  partition  the  image  into  meaningful  parts  by  using 
the  statistical  pattern  recognition  techniques.  Although  most  image 
segmentation  works  have  been  nons t a t is t i c al  in  nature,  there  is  now 
strong  interest  in  the  use  of  the  supervised  and  the  unsupervised 
classification  techniques  for  image  segmentation.  In  this  paper,  a 
critical  comparison  is  made  on  the  supervised  image  segmentation 
techniques  including  the  Fisher's  linear  discriminant,  the  auto¬ 
regressive  moving-average  modelling,  the  maximum  likelihood  region 
estimation,  and  the  maximum  a  posteriori  region  estimation,  as  well 
as  on  the  unsupervised  image  segmentation  techniques  including  the 
cluster  analysis,  the  estimation- theory  based  method,  histogram 
directed  segmentation  techniques,  and  the  decision-directed  method 
using  the  conditional  propulation  mixture  model.  Some  computer  results 
are  presented.  The  fundamental  issues  in  the  statistical  image 
segmentation  and  the  related  topics  are  also  reviewed. 
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A  Comparative  Evaluation  of  Statistical 
Image  Segmentation  Techniques 

I.  Introduction 

Statistical  image  segmentation  refers  to  computer-oriented  pro¬ 
cedures  that  partition  the  image  into  meaningful  parts  by  using 
statistical  pattern  recognition  techniques.  Although  most  image 
segmentation  work  has  been  nons t a t is t i cal  in  nature,  there  is  now 
strong  and  renewed  interest  on  the  use  of  supervised  and  unsupervised 
classification  techniques  for  image  segmentation.  In  this  semi¬ 
tutorial  paper,  the  fundamental  issues  on  statistical  image  segmenta¬ 
tion  are  examined.  A  critical  comparison  is  made  on  supervised  image 
segmentation  techniques  including  the  Fisher’s  linear  discriminant, 
the  autoregressive  moving-average  (ARMA)  modelling,  the  maximum  like¬ 
lihood  region  estimation,  and  the  maximum  a  posteriori  region  estima¬ 
tion,  as  well  as  on  the  unsupervised  image  segmentation  techniques 
including  the  cluster  analysis,  the  es timation- theory  based  method, 
histogram  directed  segmentation  techniques,  and  the  decision-directed 
method  using  the  conditional  population  mixture  model.  Some  computer 
results  are  presented.  It  is  concluded  that  the  supervised  techniques, 
whenever  applicable,  generally  can  perform  better  than  the  unsupervised 
techniques.  Presently  the  statistical  approaches  are  limited  to  seg¬ 
mentation  using  pixel  and  local  properties.  The  use  of  global  informa¬ 
tion  such  as  the  statistical  inter-region  dependence  as  well  as  the 
mixed  statistical-structural  approach  is  also  discussed. 

It  is  noted  that  an  earlier  semi-tutorial  paper  on  image  segmentation 
M  discussed  the  segmentation  techniques  from  the  standpoints  of  image 
models  that  include  both  statistical  and  spatial  models.  The  present 
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paper  is  limited  to  the  discussion  of  statistical  techniques  in  image 
segmentation . 

II.  Supervised  Image  Segmentation 

1.  Fisher’s  Linear  Discriminant  Method 

Ahuja  et,  al.  J  2j  recently  presented  detailed  computer  results 
on  the  pixel  classification  by  using  the  Fisher’s  linear  discriminant 
method.  They  use  as  features  the  gray  levels  in  the  neighborhood  of  a 
point.  Further  computer  work  performed  by  us  ^ 3 J  also  indicates  that 
the  Fisher’s  linear  discriminant  is  quite  effective  for  the  segmentation 
of  infrared  and  reconnaissance  images.  A  new  form  of  theoretical  error 
probability  is  proposed  which  compares  very  favorably  with  the 
experimental  error  rate.  For  illustrative  purpose,  Figs.  1-5  show  the 
segmentation  of  a  reconnaissance  image.  Figure  1  indicates  the  assign¬ 
ment  of  regions,  I,  II,  III  for  the  image  considered.  Figure  2a  is 
the  original  image.  All  displays  are  in  binary  (or  two-level)  form. 

A  neighborhood  of  size  3x3  is  considered  in  this  study.  Without 
additive  noise,  Fig.  2b  is  the  classified  (segmented)  result  with 
Mi ,  M2  learning  samples  taken  from  regions  II  and  I  respectively. 
Similarly  Fig.  2c  is  the  classified  result  with  M2  learning  samples 

taken  from  regions  III  and  I  respectively.  Each  learning  sample 
corresponds  to  the  gray  levels  of  a  3x3  neighborhood.  Because  of  the 
gray  level  distribution  in  region  II,  the  classified  result  given  by 
Fig.  2c  is  better  than  that  of  Fig.  2b.  Figures  3,  4,  and  5  include 
additive  white  Gaussian  noise  with  variance  1  and  zero  mean  and  with 
signal- to-noise  ratios  of  0.2,  1.0,  2.0  iespectively .  In  these  three 

sets  of  figures,  b  and  c  are  classified  results  by  using  fixed  weight 
matrix  computed  from  original  image  of  Fig.  2a,  d  and  e  are  classified 
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results  by  using  weight  matrix  computed  from  the  noisy  image  provided. 

The  experimental  results  clearly  illustrate  dependence  of  segmentation 
result  on  the  selection  of  learning  samples. 

2.  The  ARMA  modelling 

One  approach  to  model  the  spatial  dependence  of  gray  levels  in 
an  image  is  to  consider  the  image  as  a  sequence  of  random  variables, 
each  of  which  is  dependent  on  some  of  the  preceding  ones.  The  approach 
uses  the  autoregressive  moving-average  (ARMA)  model.  For  a  given  order 
of  ARMA  model,  each  region  is  characterized  by  a  set  of  model  parameters. 

A  new  image  can  be  constructed  from  the  predicted  values  of  the  ARMA 
model.  An  optimum  threshold  can  then  be  determined  to  segment  the  pre¬ 
dicted  image  into  a  binary  image.  Recently  we  employed  a  second  order 
two-dimensional  ARMA  model  for  noise  filtering  and  image  segmentation 
4j  .  For  illustrative  purpose.  Fig.  6  shows  the  result  of  segmentation 
by  ARMA  model.  Figure  6a  is  the  original  image  (same  scene  as  Fig.  2a) 
in  two-level  display  along  with  the  histogram  of  the  original  image. 

Figure  6b  is  the  noisy  image.  Figures  6c  and  6d  are  the  segmented 
images  after  noise  filtering.  Histograms  of  the  images  before  segmenta¬ 
tion  are  also  shown.  The  s ignal- to-noise  ratio  considered  is  1.73. 

The  result  is  comparable  to  that  of  Fig.  5,  but  the  ARMA  modeling  requires 
much  more  computation.  The  above  operations  are  in  fact  unsupervised. 

If  classified  learning  samples  of  object  and  background  are  given, 
better  pixel  classification  is  expected  by  using  model  parameters 
(coefficients)  as  features.  Further  work  is  needed  in  this  direction. 

The  ARMA  models  have  been  quite  useful  for  texture  analysis  and 
synthesis  ([5]  and  extensive  references  in  this  paper),  and  are  thus 


particularly  suitable  for  segmentation  of  textured  images.  The 
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limitation  of  the  models,  as  clearly  pointed  out  in  |^lj,  is  that 
symmetrical  dependence  is  mathematically  much  less  tractable. 

3.  Maximum  Likelihood  Estimation  and  Maximum  a  Posteriori  Estimation 
An  algorithm  involving  comparison  of  linear  predictive  error 
residuals  has  been  suggested  by  Haralick  j^5 J  .  This  approach  can  be 
shown  to  be  equivalent  to  computing  a  maximum  likelihood  estimate  for 
the  regions  when  the  textures  are  modeled  by  autoregressive  processes 
driven  by  white  noise  sources  with  equal  variances.  Recently  Therrien 
indicated  that  the  maximum  likelihood  procedure  tends  to  produce 
an  abundance  of  false  regions.  He  proposed  a  maximum  a  posteriori 
region  estimate  as  follows.  Observe  that  a  segmentation  of  the  image 
is  completely  equivalent  to  an  assignment  of  the  pixels  to  0  and  1. 

Such  an  assignment  for  a  pixel  is  referred  to  as  its  state  and  assume 
that  the  states  form  a  Markov  chain  with  certain  transition  probabilities. 
By  incorporating  the  transition  probabilities  in  the  probability  density 
for  the  image,  we  can  form  the  posterior  probability  of  the  state  assign¬ 
ment.  An  iterative  procedure  is  used  to  maximize  the  posterior  probabil¬ 
ity.  Experimentally  Therrien  was  able  to  show  that  the  maximum  a 
posterior  region  estimate  performs  better  than  the  maximum  likelihood 
estimate.  To  segment  the  image  into  background  and  object,  it  is  noted 
that  learning  samples  from  each  class  are  needed  in  both  maximum  like¬ 
lihood  and  maximum  a  posteriori  estimates. 

III.  Unsupervised  Image  Segmentation 
1.  Cluster  Analysis 

A  segmented  image  may  be  considered  as  a  set  of  clustered  points 
in  a  high  dimensional  space.  Each  cluster  may  possess  certain  distinctive 
characteristics  such  as  the  homogeneity  to  aid  further  analysis.  Thus 
the  clustering  methods  developed  in  the  mathematical  pattern  recognition 


provide  a  promising  approach  to  the  image  segmentation,  although  it  has 
previously  been  believed  that  a  pure  clustering  approach  was  too  cumber¬ 
some  computationally  to  implement.  Computationally  efficient  algorithms 
have  been  developed  for  image  segmentation  (e.g.  M  M  M  M  )  .  The 
basic  procedure  is  a  K-means  clustering  algorithm  which  converges  to  a 
local  minimum  in  the  average  squared  intercluster  distance  for  a  specified 
number  of  clusters.  Experimentally  we  have  found  that  the  algorithm  is 
effective  for  certain  images.  To  add  flexibility  to  the  algorithm,  the 
test  of  cluster  validity  and  reclustering  must  be  incorporated  in  the 
segmentation  process. 

Preprocessing  and  feature  selection  are  almost  always  needed.  Pre¬ 
processing  includes  image  smoothing  and  sharpening  operation,  decorrela¬ 
tions,  etc.  while  the  features  selected  may  relate  to  the  gray-level 
(brightness)  and  texture  for  several  window  sizes  centered  on  every 
pixel.  Feature  evaluation  and  reduction  can  make  use  of  the  distance 
measures.  For  a  discussion  of  the  texture  features,  see  W  M  It 
is  noted  that  the  clustering  analysis  is  an  unsupervised  approach  to 
segmentation.  Even  though  such  segmentation  may  not  be  as  satisfactory 
as  human  segmentation,  it  is  necessary  in  some  practical  applications 
in  which  the  classified  learning  samples  may  not  be  available  or  the 
real-time  segmentation  is  required.  The  unsupervised  approach  may  also 
reveal  some  characteristics  in  the  image  that  were  unobserved  by  human 
observers . 

2.  Estimation- theory  Based  Method 

Image  segmentation  can  be  considered  as  an  estimation  problem. 

Chen  and  Pavlidis  proposed  an  estimation-decision  process  within 

the  framework  of  a  spli t-and-merge  algorithm.  After  statistical  parameter 


estimation,  a  region  can  be  tested  to  determine  its  uniformity,  i.e. 
homegeneity.  S p li t- and-me rge  algorithm  can  be  applied  to  grow  or  split 
regions  as  needed.  After  a  sufficiently  large  number  of  iterations, 
small  regions  containing  the  boundary  will  remain.  Curve  fitting  method 
can  then  be  used  to  determine  the  boundary  accurately.  For  the  textured 
images,  the  authors  also  considered  the  use  of  correlation  coefficients 
for  segmentation  £ 1 2  J  in  terms  of  the  spli t-and-merge  algorithm. 

3.  Histogram  Directed  Segmentation 

In  spite  of  many  recent  research  efforts  in  image  segmentation, 
the  simple  histogram  directed  segmentation  remains  to  be  an  effective 
segmentation  method  [l^J  as  compared  to  the  other  methods  such  as  the 
quadtree  based  method  and  the  relaxation  method.  The  his  tog r am-di r e c t ed 
segmentation  may  also  be  used  as  an  initial  segmentation  which  can  be 
refined  with  other  methods.  The  optimum  threshold  must  be  determined 
for  the  one-dimensional  and  multivariate  histogram  directed  segmentation 
(see  e  .  g.  [l4]  )  . 

4.  Conditional  Population-Mixture  Model 

The  conditional  population-mixture  model  as  suggested  by  Sclove 
[»j  is  a  decis ion-di rec ted  image  segmentation  or  clustering  technique. 
The  K-means  clustering  algorithm  is  a  special  case  of  the  model.  By 
using  a  2x2  image  sample,  the  4- dimens  ion al  feature  space  may  be  assumed 
to  be  multivariate  Gaussian  with  the  parameters  iteratively  estimated 
by  decision  directed  learning  samples.  Assume  that  the  background  and 
object  are  two  regions  under  consideration.  As  the  mean  vectors  and 
cova riant  matrices  are  different  for  the  two  regions,  the  complete 
probability  density  rather  than  the  Euclidean  distance  or  weighted 
distance  should  be  used  for  pixel  classification.  We  have  performed 
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experimental  study  of  this  method  for  comparison  with  other  statistical 
image  segmentation  methods.  Figure  7b  is  the  noisy  image  of  a  portion 
of  airport  scene  (Fig.  7a)  studied  in  Figs.  1-6.  For  the  signal-to- 
noise  ratio  of  1.73  considered  the  decision  directed  segmentation  result 
is  shown  in  Fig.  7c.  If  the  s ignal- to-no ise  ratio  is  reduced  to  0.87 
as  shown  in  Fig.  7d,  the  segmented  result  as  shown  in  Fig.  7e  has  more 
noise.  If  the  noisy  image  is  first  preprocessed  with  median  filtering 
and  then  by  the  conditional  population-mixture  model,  the  result  is 
much  improved  as  shown  in  Fig.  7f.  Generally  speaking  the  segmented 
result  is  noisier  than  those  of  Fisher's  linear  discriminant  and  the 
ARMA  model.  Furthermore  the  method  requires  considerably  more  computation 
and  the  convergence  is  slow. 

IV.  Review  of  the  Fundamental  Issues 

The  most  fundamental  issue  in  the  statistical  image  segmentation  as 
well  as  class i f i ca t ion  is  the  use  of  contextual  information  [l6j  .  It  is 
not  possible  in  most  cases  to  obtain  the  probability  density  of  a  set  of 
neighborhood  pixels  without  making  simplifying  assumption  such  as  Markov 
dependence  and  ARMA  modelling.  Various  methods  of  the  statistical  image 
segmentation  have  attempted  to  utilize  the  contextual  information  in 
the  computationally  feasible  ways.  In  image  analysis  we  often  have  too 
limited  number  of  samples  for  the  statistical  analysis  to  provide  a 
reliable  segmentation.  As  discussed  in  the  previous  sections,  good 
statistical  descriptions  of  various  regions  are  needed  for  effective 
segmentation.  Iterative  procedures  are  almost  always  used  to  establish 
accurate  statistical  information.  Proper  selection  of  image  features 
is  another  problem  area  closely  related  to  Lhe  statistical  segmentation. 
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V.  Related  Topics 

Many  computer  results  have  shown  that  the  statistical  image  segmenta¬ 
tion  is  very  promising.  There  are  fundamental  limitations  on  what  the 
statistical  methods  can  accomplish.  Presently  the  statistical  approaches 
are  limited  to  the  segmentation  using  pixel  and  local  properties  as  it 
is  much  easier  to  establish  the  statistical  information  of  a  pixel  and 
its  neighborhood.  Such  local  statistical  information  is  indeed  important 
but  not  sufficient  for  the  more  complete  characterization  of  an  image. 
Statistical  dependence  among  regions  should  be  incorporated  in  the 
segmentation  process.  A  simple  technique  is  to  use  the  correlation 
coefficient  to  test  the  similarity  among  regions. 

It  is  important  to  note  that  the  image  also  contains  structural 
information  which  cannot  be  properly  described  statistically.  How  to 
utilize  both  statistical  and  structural  information  in  an  effective 
manner  has  been  a  difficult  problem.  However,  some  promising  approach 
has  now  been  available  [17]  and  it  has  confirmed  that  the  mixed 
statistical-structural  approach  can  perform  better  than  the  use  of  the 
statistical  or  the  structural  approach  alone. 

Finally  we  would  like  to  point  out  that  a  number  of  important  works 
have  been  done  at  the  edge  (boundary)  based  statistical  segmentation. 

This  is  a  proper  subject  for  a  separate  paper. 
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Statistical  image  segmentation.  Fisher's  linear  discriminant  method. 

The  ARMA  modelling.  Cluster  analysis.  Estimation-theory  image  segmentation. 
Conditional  population-mixture  model  for  image  segmentation. 

Supervised  and  unsupervised  image  segmentation. 
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Statistical  image  segmentation  refers  to  the  computer-oriented  procedures 
that  partition  the  image  into  meaningful  parts  by  using  the  statistical 
pattern  recognition  teclnniques.  Although  most  image  segmentation  works  have 
been  ncnstatistical  in  nature,  there  is  new  strong  interest  in  the  use  of  the 
supervised  and  the  unsupervised  classification  techniques  for  image  segmentation. 
In  this  paper,  a  critical  comparison  is  made  on  the  supervised  image 
segmentation  techniques  including  the  Fisher's  linear  discriminant,  the 
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autoregressive  moving-average  modelling,  the  maximum  likelihood  region 
estimation,  and  the  maximum  a  posteriori  region  estimation,  as  well  as  cn 
the  unsupervised  image  segmentation  techniques  including  the  cluster  analysis, 
the  estimation -theory  based  method,  histogram  directed  segmentation  techniques 
and  the  decision-directed  method  using  the  conditional  population  mixture 
model.  Seme  computer  results  are  presented.  The  fundamental  issues  in  the 
statistical  image  segmentation  and  the  related  topics  are  also  reviewed. 


